<!DOCTYPE html 
     PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<title>The Java Faster than C++ Benchmark</title>
<style type="text/css">
code.cmd { background: #ffa }
</style>
</head>
<body>
<p> <a href="data"><img src="graph-small" width="200" height="172" style="border: 2px solid; float: right; margin: 5px;" alt="Performance Graph" /></a> </p>

<h1> <a name="intro" /> The Java is Faster than C++ and C++ Sucks Unbiased Benchmark </h1>

<h2> Introduction </h2>
<p> I was sick of hearing people say Java was slow, when I know it's pretty fast, so I took the benchmark code for C++ and Java from the now outdated <a href="http://www.bagley.org/~doug/shootout/">Great Computer Language Shootout</a> and ran the tests myself. </p>

<p> I used <i>G++ (GCC) 3.3.1 20030930</i> (with <i>glibc 2.3.2-98</i>) for the C++, with the <code>-O2</code> flag (for both <code>i386</code> and <code>i686</code>). I compiled the Java code normally with the Sun <i>Java 1.4.2_01</i> compiler, and ran it with the Sun 1.4.2_01 JVM. I ran the tests on <i>Red Hat Linux 9 / Fedora Test1</i> with the <i>2.4.20-20.9</i> kernel on a T30 laptop. The laptop has a Pentium 4 mobile chip, 512MB of memory, a sort of slow disk. </p>

<p> The results I got were that Java is significantly faster than optimized C++ in many cases. </p>

<p> The results I got show that no one should ever run the client JVM when given the choice. (Everyone has the choice. To run the server VM, see instructions in the <a href="#servervm">Using the Server JVM</a> section below.) </p>

<p> I will post here anyone else's results as long as they use Java 1.4.2 or higher and any version of GCC that produces faster or equivalent code than the 3.3.1 I used. I encourage you to download the source and/or the binaries and perform the tests yourself, with your favorite compiler and on your favorite platform. See the <a href="#notes">Notes</a> section below for details. </p>

<h2> <a name="data" /> Data and Results </h2>
<ul>
<li> <a href="data">Data table and graph</a> </li>
<li> <a href="runlog">Console run log</a> </li>
<li> Source:
<ul>
<li> <a href="src/cpp/">C++ code</a> </li>
<li> <a href="src/java/">Java code</a> </li>
</ul>
</li>
<li> Binaries
<ul>
<li> C++ binaries (Linux)
<ul>
<li> <a href="bin/cpp/i386/">Intel 386 (Basic Pentium or lower)</a> </li>
<li> <a href="bin/cpp/i686/">Intel 686 (Optimized for P6-class - Pentium Pro/II, Celeron 266-533MHz, original Athlon, higher) </a> </li>
</ul>
</li>
<li> <a href="bin/java/">Java class files</a> </li>
</ul>
</li>
<li> Download all source code and binaries: <a href="javabench-src.tar.bz2" title="Java Faster than C++ Benchmark Code">.bz2</a>, (69KB) <a href="javabench-src.zip" title="Java Faster than C++ Benchmark Code">.zip</a> (189KB) </li>
</ul>

<h2> <a name="notes" /> Notes </h2>

<p> JVM startup time <i>was</i> included in these results. That means even with JVM startup time, Java is still faster than C++ in many of these tests. </p>

<p> Some of the C++ tests would not compile. I've never been very good at decoding GCC's error messages, so if I couldn't fix a test with a trivial modification, I didn't include it in my benchmarks. </p>

<p> I modified one of the tests, the string concatenation test for Java. The test was creating a new <code>StringBuffer</code> in each iteration of the loop, which was just silly. I updated the code to use a single StringBuffer and appending to it inside the loop. (The updated tests at the original shootout use this new method.) <b><i>Java lost this benchmark even with the modifications,</i></b> so if you want to accuse me of biasing the results, you're going to have to try harder. </p>

<p> Several versions of some of the C++ tests (like <code>matrix</code>) were present in the original shootout source. I used the versions without numbers in them, like <code>matrix.g++</code> instead of <code>matrix.g++2</code>. I don't know which of these were used in the original benchmarks, but from my quick experimenting, the numberless ones generally ran faster than their numbered counterparts. (Looking at them again, <code>matrix.g++3</code> runs faster than the <code>matrix.g++</code> that I use. However, it still runs slower than the Java version, so I don't plan to modify the graph/data unless someone asks me to, since getting that graph in the first place was sort of a pain.) </p>

<p> I've been told that the C++ code for the Method Call benchmark returns by value while the Java code returns by reference, and that modifying the C++ code to pass a pointer makes that benchmark faster. However, even with the modification, the C++ version still runs slower than the Java version. </p>

<p> I ran the tests many times before running the "official" recorded set of tests, so there was plenty of time for both Java and the C++ tests to "warm up" (both the Java and C++ tests got faster after I ran them a few times). </p>

<p> I've been told that these tests are invalid because they were run with GCC. I have seen both <a href="http://www.aceshardware.com/read.jsp?id=153">benchmarks</a> that show GCC producing faster code than Visual Studio's VC++ compiler, and benchmarks showing the opposite. If I update the benchmarks with another compiler added, it will be the Intel C++ Compiler, which I'm pretty sure produces faster code than VC++. </p>

<p> I've been accused of biasing the results by using the <code>-O2</code> option for GCC, supposedly because -O2 optimizes for space, thus slowing down the benchmark. <b>This is not what -O2 does.</b> According to the <a href="http://gcc.gnu.org/onlinedocs/gcc-3.3.2/gcc/Optimize-Options.html" title="GCC optimization flags documentation">GCC -O documentation</a>:
<blockquote>
-O2: Optimize even more. GCC performs nearly all supported optimizations that do not involve a space-speed tradeoff. The compiler does not perform loop unrolling or function inlining when you specify -O2. As compared to -O, this option increases both compilation time and the performance of the generated code.
</blockquote> On the other hand, -O3 performs space-speed tradeoffs, and -O performs fewer optimizations. Thus, for these tests, I think -O2 was the best choice. </p>

<p> I don't have an automated means of building and benchmarking these things (and the scripts that came with the original shootout didn't run for me). I really do want you to test it on your own machine, but it's going to take some work, I guess. I compiled the C++ code with:</p>
<blockquote>
<p><code class="cmd">g++ <i>[test]</i>.cpp -O2 -march=i386 -o <i>[test]</i>-386</code></p>
<p><code class="cmd">g++ <i>[test]</i>.cpp -O2 -march=i686 -o <i>[test]</i>-686</code></p>
</blockquote>
<p>I compiled the Java code with:</p>
<blockquote>
<p><code class="cmd">javac <i>[test]</i>.java</code></p>
</blockquote>
<p> To see how I ran the binaries, see the <a href="runlog">run log</a>. You can download the source code I used in <a href="javabench-src.tar.bz2" title="Java Faster than C++ Benchmark Code">.bz2</a>, <a href="javabench-src.zip" title="Java Faster than C++ Benchmark Code">.zip</a> format. </p>

<h2> <a name="servervm" /> Using the Server JVM </h2>

<p> Every form of Sun's Java runtime comes with both the "client VM" and the "server VM." Unfortunately, Java applications and applets run by default in the client VM. The Server VM is much faster than the Client VM, but it has the downside of taking around 10% longer to start up, and it uses more memory.</p>

<p> There are two ways to run Java applications with the server VM: </p>
<ol>
<li> When launching a Java application from the command line, use <code class="cmd">java&nbsp;-server&nbsp;<i>[arguments...]</i></code> instead of <code class="cmd">java&nbsp;<i>[arguments...]</i></code>. For example, use <code class="cmd">java&nbsp;-server&nbsp;-jar&nbsp;beanshell.jar</code>. </li>
<li> Modify the <code>jvm.cfg</code> file in your Java installation. (It's a text file, so you can use Notepad or Emacs to edit it.) This is located in <i>C:\Program Files\Java\j2reXXX\lib\i386\</i> on Windows, <i>/usr/java/j2reXXX/lib/i386/</i> on Linux. You will see two lines:
<blockquote>
<pre>-client KNOWN
-server KNOWN</pre>
</blockquote>
You should change them to:
<blockquote>
<pre>-server KNOWN
-client KNOWN</pre>
</blockquote>
This change will cause the server VM to be run for all applications, unless they are run with the <code>-client</code> argument.
</li>
</ol>

<h2> Contact </h2>

<p> I can be contacted at <a href="mailto:keith@kano.net">keith@kano.net</a>. </p>

<h2> Links </h2>

<ul>
<li> <a href="http://members.lycos.co.uk/wjgoh/JavavsC.html">Java vs. C benchmark</a> - the benchmark is in Microsoft Word format </li>
<li> <a href="http://research.sun.com/techrep/2002/abstract-114.html">FreeTTS performance case study</a> - at Sun Research </li>
<li> <a href="http://www.coyotegulch.com/reviews/almabench.html">Linux Number Crunching</a> - used 1.4.1 VM, got very different results than I did, maybe due to floating point optimizations added in 1.4.2 </li>
<li> <a href="http://home.fnal.gov/~kuropat/snap/SNAP.html">SNAP matrix benchmark</a> - Java vs. C++ for matrix multiplication </li>
<li> <a href="http://osnews.com/story.php?news_id=5602">Nine-Language Performance Round-Up</a> - math and file I/O performance among nine languages </li>
<li> <a href="http://www.tommti-systems.de/main-Dateien/reviews/languages/benchmarks.html">Performance comparison C++, C# and Java</a> - similar results as mine, mostly </li>
<li> <a href="http://burk.hax.se/~nils/javavsc_eng.html">Java vs. C</a> - for bubble sort, IBM's 1.3.0 client JVM comes out on top over GCC with full optimizations</li>
<li> <a href="http://www.aceshardware.com/Spades/read.php?article_id=154">Runtime Vs Static Compilation - Performance Comparison</a> - even old versions of Java can run fast </li>
<li> <a href="http://cpp.student.utwente.nl/benchmark">Comparing C++ performance with Java</a> - these are my benchmarks, except more hand-optimized and compiler-optimized (Java wins about half of them) </li>
<li> <a href="http://www.idiom.com/~zilla/Computer/javaCbenchmark.html">Java Pulling Ahead?</a> - "finds that Java performance on numerical code is comparable to that of C++, with hints that Java's relative performance is continuing to improve" </li>
<li> <a href="http://www.laboiteaprog.com/tutoriel725">Benchmark C++ vs JAVA</a> - ran my benchmarks and added GCJ and Java 5.0 results </li>
<li> <a href="http://www.w3sys.com/pages.meta/benchmarks.html">C++ vs Java</a> - David K. McCombs doesn't like my measurement of CPU time and some of the benchmark's C++ code </li>
<li> <a href="http://www.freewebs.com/godaves/javabench_revisited/">'The Java Faster than C++' Benchmark Revisited</a> - someone doesn't like my benchmarks </li>
<li> <a href="http://www.shudo.net/jit/perf/">Performance Comparison of Java/.NET Runtimes</a> - lots of VM's compared</li>
<li> <a href="http://bruscy.multicon.pl/pages/przemek/java_not_really_faster_than_cpp.html">The Java (not really) Faster than C++ Benchmark</a> - another benchmark with hand-optimized C++ code and different gcc flags</li>
<li><a href="http://www-128.ibm.com/developerworks/java/library/j-jtp09275.html?ca=dgr-jw22JavaUrbanLegends">Java Performance Urban Legends</a> by Brian Goetz, discusses garbage collection vs. manual allocation &amp; deallocation</li>
</ul>

<!--Creative Commons License--><a rel="license" href="http://creativecommons.org/licenses/by-sa/2.5/"><img alt="Creative Commons License" border="0" src="http://creativecommons.org/images/public/somerights20.png"/></a><br/>This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-sa/2.5/">Creative Commons Attribution-ShareAlike 2.5 License</a>.<!--/Creative Commons License--><!-- <rdf:RDF xmlns="http://web.resource.org/cc/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<Work rdf:about="">
<license rdf:resource="http://creativecommons.org/licenses/by-sa/2.5/" />
<dc:type rdf:resource="http://purl.org/dc/dcmitype/Text" />
</Work>
<License rdf:about="http://creativecommons.org/licenses/by-sa/2.5/"><permits rdf:resource="http://web.resource.org/cc/Reproduction"/><permits rdf:resource="http://web.resource.org/cc/Distribution"/><requires rdf:resource="http://web.resource.org/cc/Notice"/><requires rdf:resource="http://web.resource.org/cc/Attribution"/><permits rdf:resource="http://web.resource.org/cc/DerivativeWorks"/><requires rdf:resource="http://web.resource.org/cc/ShareAlike"/></License></rdf:RDF> -->
<script src="http://www.google-analytics.com/urchin.js" type="text/javascript">
</script>
<script type="text/javascript">
_uacct = "UA-95484-1";
urchinTracker();
</script>
</body>
</html>
